An Exact Implicit Enumeration Algorithm for Variable Selection in Multiple Linear Regression Models Using Information Criteria

For large multivariate data sets the data analyst often wants to know the best set of independent regressors to use in a multiple linear regression model. Akaike’s Information Criteria (AIC) is one information criterion calculated in SAS that is used to score a model. For a small number of independent variables p, an explicit enumeration of all possible 2 models is possible. However, for large multivariate data sets where p is large, an explicit enumeration of all possible models becomes computationally intractable. This paper presents SAS code that implements the exact implicit enumeration algorithm authored by Bao (2005) that has been shown to always arrive at the globally optimal minimum AIC value when let run to completion. The number of models evaluated to determine the optimal model with the smallest AIC score is minimal and shown to be much more efficient than an explicit enumeration of all possible models. A large multivariate data set is simulated with a known true model to demonstrate how fast the exact implicit enumeration algorithm arrives at the true model. The number of models evaluated is compared to an explicit enumeration algorithm and the REG procedure in SAS. This paper is for intermediate SAS users of SAS/STAT who understand multivariate data analysis and SAS macros.